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ABSTRACT 

The  purpose  of  this  thesis  is  to  determine  if  format 
variation  has  any  affect  on  speed  and  accuracy  of  data 
entry  for  the  Marine  Integrated  Fire  and  Air  Support  System 
(MIFASS) . 

Independent  variables  for  the  test  are  message  formats, 
message  types,  and  subjects.   Subject  learning  effects  and 
experimenter  differences  are  controlled  by  use  of  a  pilot 
study.   Subjects  are  included  as  a  treatment  in  order  to 
estimate  their  effects. 

The  experiment  is  designed  to  support  both  parametric 
and  nonparametric  analyses.   A  4x5x8  randomized  complete 
block  factorial  with  two  replications  is  used  to  analyse 
the  effects  of  format  variation.   Friedman's  Two-Way  ANOV 
by  Ranks  is  used  to  test  subjective  rankings  of  the  formats 
as  concerns  ease  of  use  and  operational  suitability. 

It  is  concluded  that  format  variations  do  affect  speed 
of  data  entry  but  not  accuracy.   Format  3  appears  to  be 
superior.   Recommendations  for  further  study  are  given. 
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I.   STATEMENT  OF  THF  PROBLEM 

A.   THE  PROBLEM  IN  GENERAL 

The  primary  function  of  a  Fire  Support  Coordination 
Center  (FSCC)  at  any  level  of  command  is  to  provide  the 
commander  with  the  means  for  planning  and  coordinating 
available  fire  support.   Specific  responsibilities  of  a 
FSCC  include  fire  support  planning,  designation  of  targets 
and  assignment  of  priorities,  allocation  of  fire  support 
resources,  and  coordination  of  supporting  arms  with  each 
other  and  the  supported  force.   At  present,  all  these 
responsibilities  and  functions  are  carried  out  manually 
through  the  use  of  radio  and  telephone  communications, 
personal  liaison,  and  an  immense  amount  of  paperwork.   An 
inordinate  amount  of  valuable  time  is  lost  in  the  FSCC 
during  normal  operations  because  of  the  paperwork  and 
liaison  required  to  accomplish  its  mission.   In  an  attempt 
to  eliminate  this  bottleneck  and  more  effectively  use  the 
skill  and  training  of  the  personnel  who  operate  the  FSCC, 
the  Marine  Corps,  in  conjunction  with  Hughes  Aircraft 
Company,  is  developing  an  automated  system  which  will 
materially  reduce  the  paperwork,  list  keeping,  and  manual 
labor  presently  required  in  a  FSCC.   The  system  will  be 
called  the  Marine  Integrated  Fire  and  Air  Support  System 
(MIFASS) . 


B.   OVERVIEW  OF  THE  AUTOMATED  FSCC 

The  concept  for  MIFASS  operations  offers  extensive 
improvement  in  tactical  effectiveness  through  automation  of 
fire  request  planning,  fire  support  coordination,  fire  and 
air  support  planning,  conflict  prediction,  and  the  consoli- 
dation of  many  other  common  functions  at  the  MIFASS  centers. 
The  automated  system  will  utilize  various  configurations  of 
the  hardware  listed  in  Table  I  at  designated  levels  of  the 
operational  tactical  unit.   Subunits  operating  under  tacti- 
cal control  of  the  MIFASS  centers  will  input  intelligence 
data,  fire  support  requests,  med-evac  requests,  damage 
assessment  reports,  etc.,  to  the  MIFASS  computer.   MIFASS 
software  will  provide  rapid  response  to  many  on-line  users 
by  interleaving  the  processing  tasks  and  overlapping  the 
input/output  (I/O)  demands  with  processing.   Through 
generalized  data  management  software,  the  system  will  permit 
definition  of  new  messages,  files,  reports,  and  displays  in 
the  field  without  reprogramming.   The  application  programs 
envisioned  will  include  computer  assistance  and  algorithm 
development  for  major  requisite  functions  such  as  fire 
mission  analysis,  target  data  listing,  air  support  control, 
technical  fire  control,  conflict  detection,  troop  safety, 
mission  scheduling,  resource  allocation,  etc.   Since  each 
FSCC  will  have  access  to  the  memory  bank  and  coordinating 
agencies  will  be  able  to  view  the  same  data  simultaneously 
on  their  own  output  units,  the  time  needed  to  perform  the 
required  functions  for  all  subunits  will  be  materially 
decreased  thereby  increasing  the  effectiveness  of  the 
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overall  system.   Due  to  the  probability  that  several  agencies 
will  be  viewing  the  same  data  simultaneously  and  in  the 
interests  of  simplification  in  training  and  programming, 
it  is  evident  that  a  simplified  and  uniform  data  I/O  pro- 
cedure will  be  required. 

TABLE  I 
HARDWARE  REQUIRED  FOR  A  FIELDED  MIFASS 

Each  FSCC  will  employ  a  specific  configuration  of  some  or 
all  of  these  units. 

Digital  Message  Entry  Device  (DMED) 

Portable  Printer 

Remote  Terminal  Device  (RTD) 

Cathode  Ray  Tube  (CRT)  Display 

Small  Auxiliary  Memory 

Hard  Copy  Printer 

Keyboard  Entry  Panel 

Communications  Net 
Computer  with  Internal  Memory  16-131K 
Auxiliary  Mass  Memory  500K-8M  Characters 
Magnetic  Tape  Units 
Printers 
Plotters 

Large  Screen  Displays 
Textual/Tabular  Displays 
Graphic  Displays 
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C.  DATA  ENTRY  PROCEDURES 

Proper  data  I/O  procedures  are  crucial  to  the  function- 
ing of  an  automated  system.   One  phase  of  these  procedures 
which  presents  a  potential  bottleneck  to  the  system  is  the 
format  used  when  entering  data  or  outputting  it  on  a  grid 
display  or  copier  for  viewing  by  interested  parties.   If  a 
uniform  format  is  not  used  by  everyone,  users  will  encounter 
problems  in  data  retrieval  and  in  interpretation  of  data 
entered  by  other  on-line  units.   The  requirement  for  a 
format  (or  formats)  understood  by  all  which  is  fast,  accu- 
rate, and  convenient  is  obvious.   Determining  the  feasi- 
bility of  such  a  format  was  the  overall  objective  of  this 
experiment. 

D.  PURPOSE  OF  THE  TEST 

As  one  approach  to  determining  the  feasibility  of  a  uni- 
form format  for  use  in  the  system,  this  test  was  designed 
to  evaluate  differences  in  operator  data  entry  speed  and 
accuracy  resulting  from  the  use  of  alternative  message 
formats  when  used  with  differing  types  of  I/O  data.   The 
convenience  and  operational  suitability  of  the  formats 
tested  was  to  be  determined  through  operator  surveys. 

E.  HYPOTHESES  THAT  WERE  TESTED 

H  , :  There  is  no  significant  difference  in  the  speed  of 
data  entry  due  to  the  use  of  different  message  formats. 

H  - :  There  is  no  significant  difference  in  the  accuracy 
of  data  entry  due  to  the  use  of  different  message  formats. 
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H  ~:  There,  is  no  significant  difference  in  the  speed  of 
data  entry  due  to  the  use  of  different  message  types. 

H  .:  There  is  no  significant  difference  in  the  accuracy 
of  data  entry  due  to  the  use  of  different  message  types. 

H  £. :  There  is  no  significant  difference  in  the  speed  of 
data  entry  due  to  the  subjects  used  for  the  test. 

H  '.      There  is  no  significant  difference  in  the  accuracy 

of  data  entry  due  to  the  subjects  used  for  the  test. 

H  _:   There  is  no  significant  interaction  between  message 
o/ 

type  and  format  for  speed  of  data  entry. 

HnR:  There  is  no  significant  interaction  between  message 
type  and  format  for  accuracy  of  data  entry. 

H  ~:   There  is  no  significant  interaction  between  message 

format  and  subject  for  speed  of  data  entry. 

H  ..  A :   There  is  no  significant  interaction  between 
olO 

message  format  and  subject  for  accuracy  of  data  entry. 

H  , , :   There  is  no  significant  interaction  between 
oil 

message  type  and  subject  for  speed  of  data  entry. 

H  ,  ~:   There  is  no  significant  interaction  between 
Ol2 

message  type  and  subject  for  accuracy  of  data  entry. 

H  ,  ,, :   There  are  no  significant  interactions  among 
ol3 

message  type,  message  format,  or  subject  for  speed  of  data 

entry. 

H  ,.:   There  are  no  significant  interactions  among 

message  type,  message  format,  or  subject  for  accuracy  of 

data  entry. 

H  , _ :   The  subjects  are  indifferent  to  the  message  for- 
Ol5  J 

mats  concerning  ease  of  use. 
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H  ,,:   The  subjects  are  indifferent  to  the  message  for- 
0I6  J 

mats  concerning  operational  suitability. 

F.   ASSUMPTIONS 

1.  Message  Types 

The  message  types  selected  for  this  experiment  were 
assumed  representative  of  the  total  population  of  messages 
envisioned  for  use  in  an  operational  MIFASS.   The  messages 
chosen  varied  in  length  from  62  to  573  characters  which  were 
a  mixture  of  numeric  and  alpha-numeric;  therefore,  they 
covered  a  wide  span  of  the  total  population. 

2 .  Subjects 

The  subjects  used  for  this  test  were  assumed  repre- 
sentative of  the  total  population  of  personnel  available  to 
operate  a  fielded  system.   This  assumption  might  be  ques- 
tioned in  view  of  the  experience  and  proficiency  gained  by 
the  subjects  during  the  preceding  several  months  of  system 
development.   However,  circumstances  dictated  that  these 
particular  subjects  be  used,  and,  in  any  case,  it  is  a 
certainty  that  personnel  assigned  to  operate  a  fielded 
system  will  undergo  several  weeks  of  intensive  training 
before  being  permitted  to  begin  actual  operational  duties. 
Thus,  this  assumption  was  considered  justified. 

3.  Message  Format  Effects 

Effects  of  different  message  formats  on  the  speed 
and  accuracy  of  data  entry  under  test  conditions  were  con- 
sidered typical  (ignoring  stress,  fatigue,  etc.)  of  the 
effects  which  will  be  prevalent  during  actual  operations. 
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If  it  is  subsequently  determined  that  any  conditions  make 
this  assumption  invalid,  this  experiment  can  be  repeated 
under  those  conditions  during  later  phases  of  MIFASS 
development. 

G.   GENERAL  APPROACH 

1.  The  Test 

Four  test  runs  were  made,  each  of  which  consisted  of 
a  group  of  subjects  entering  data  from  five  message  types 
into  a  specific  blank  message  format  which  was  presented  on 
a  grid  display  screen.   Each  subject  entered  data  twice  for 
each  message  type  for  a  total  of  10  trials  per  run  per 
subject.   Speed  and  accuracy  measures  were  taken  for  each 
trial.   Upon  completion  of  each  run,  the  subjects  each  com- 
pleted a  subjective  evaluation  form  concerning  the  ease  of 
use  and  operational  suitability  of  that  format. 

2 .  The  Analysis 

A  randomized  block  factorial  analysis  of  variance 
was  used  to  test  the  first  14  hypotheses.   The  two  subjec- 
tive hypotheses  were  tested  using  nonparametric  test 
procedures . 

3.  Remarks 

It  was  not  the  objective  of  this  experiment  to  choose 
a  single  format  for  use  in  a  fielded  system  but  to  test  the 
feasibility  of  such  a  format.   However,  if  one  of  the  for- 
mats tested  were  to  be  found  faster  and  more  accurate  than 
the  others  under  all  test  conditions,  it  might  be  assumed 
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that  that  format  would  be  a  good  possibility  for  adoption 
as  a  universal  format  for  the  MIFASS. 
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II.   SELECTION  OF  DESIGN 

A.   DEPENDENT  VARIABLES 

1.  Speed  of  Data  Entry 

The  time,  T.,  required  to  enter  each  stimulus 
message  into  a  blank  message  format  was  collected  for  each 
subject  during  each  experimental  run.   This  response  was 
then  normalized  by  dividing  each  T.  by  the  total  number  of 
keying  actions  required  for  that  message  format  including 

« 

any  header  data  or  spaces  over  which  the  operator  had  to 
space  or  tab  manually.   Thus,  the  dependent  variable  speed 
in  this  test  was  the  normalized  data  entry  time. 

2 .  Accuracy  of  Data  Entry 

The  accuracy  of  data  entry  was  defined  in  terms  of 
data  elements  which  are  entire  sets  of  one  or  more  charac- 
ters to  be  entered  into  a  single  message  field.   The  number, 
A. ,  of  data  elements  that  had  been  entered  incorrectly 
was  collected  for  each  message  entered  by  each  subject  dur- 
ing each  experimental  run.   Any  error  such  as  single  or 
multiple  typographical  errors  within  a  data  element,  an 
omitted  data  element,  or  one  data  element  substituted  for 
another  was  counted.   This  response  was  normalized  by 
dividing  A.  by  the  total  number  of  data  elements  that  were 
entered  in  the  i    message.   Thus,  the  dependent  variable 
accuracy  in  this  test  was  the  normalized  data  entry  accuracy 
score. 
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3.   Subjective  Rankings 

To  forestall  the  possibility  that  individual  memories 
might  affect  the  ranking  of  the  formats  if  the  subjective 
opinions  were  to  be  gathered  after  all  the  formats  had  been 
presented,  it  was  considered  necessary  to  gather  a  separate 
opinion  from  each  subject  for  each  format  immediately  after 
it  had  been  tested.   A  graduated  interval  scale  was  designed 
for  each  subjective  dependent  variable.   Each  scale  con- 
sisted of  20  intervals  with  the  endpoints  and  some  inter- 
mediate intervals  marked  with  verbal  descriptions  for  ease 
in  marking  as  shown  in  Fig.  1.   These  scales  were  then  used 
for  gathering  the  individual  opinions. 

a.  Ease  of  Use 

The  graduated  scale  used  by  each  subject  to  give 
his  opinion  of  the  ease  of  use  of  each  format  was  marked 
from  0  to  20  where  0  coincided  with  the  rating  "Easy  to  Use" 
and  20  matched  "Unusable." 

b.  Operational  Suitability 

The  opinions  for  this  subjective  dependent 
variable  were  marked  on  a  graduated  scale  where  0  coincided 
with  "Very  Suitable"  and  20  with  "Very  Unsuitable." 

B.   INDEPENDENT  VARIABLES 
1 .   Message  Formats 

The  independent  variable  of  primary  interest  was 
message  format.   There  were  four  different  formats  tested 
in  this  experiment.   Appendix  A  presents  the  actual  formats 
tested  for  each  message  type. 
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SUBJECT  REACTION  FORM 


Subject  Format 

Experimenter  Date 


Please  indicate  your  reaction  to  the  format  you  have  just 
completed  testing  by  placing  a  checkmark  at  the  appropriate 
place  on  the  line  below. 

1.  How  easy  or  difficult  was  it  to  use  this  format? 

0              5              10             15             20 
i   ■   i ■   ■ i  .  i i i i l i i i i t i i   i,i i 

Easy      Convenient      Useable      May  be       Unuseable 
to  Use  Useable 

2.  How  would  you  rate  this  format  as  far  as  suitability 
for  use  in  an  operational  MIFASS? 

0  5  10  15  20 

i i i i i i i i I    ■    ' I    ■ '    ■    '    ' I l > 1 

Very       Suitable       Possibly     Unsuitable      Very 
Suitable  Suitable  Unsuitable 


Figure  1.   Subject  Reaction  Form.   This  form  was  used  to 
gather  data  necessary  to  the  nonparametric 
analyses  required. 
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a.  Linear  Format 

In  this  format,  each  field  name  in  the  message 
was  immediately  followed,  on  the  same  line,  by  a  space  or 
spaces  in  which  the  pertinent  message  data  for  that  field 
was  to  be  entered.   One  field  followed  the  previous  field 
on  the  same  line  without  spacing  between  fields  until  that 
line  was  filled.   The  succeeding  fields  were  positioned  in 
lines  as  stated  until  the  entire  message  had  been  completed. 

b.  Block  Format 

The  block  format  had  the  field  names  in  a  linear 
sequence  with  the  message  data  space  or  spaces  immediately 
below  the  field  name  on  the  next  line  of  the  display  field. 
The  field  names  were  separated  sufficiently  to  preclude 
information  overlap  in  the  message  data  spaces  below.   No 
space  appeared  between  the  line  of  field  names  and  the  line 
containing  related  message  data.   A  double  space  appeared 
between  each  adjacent  pair  of  lines.   This  form  was  followed 
until  enough  line  pairs  appeared  to  complete  the  entire 
message. 

c.  Columnar/Separated  Format 

For  this  format,  all  field  names  appeared  in  a 
single  column  on  the  left  side  of  the  display  area.   The 
space  or  spaces  for  the  pertinent  message  data  for  each 
field  appeared  in  a  second  column  indented  from  the  left 
margin  of  the  display  area,  not  from  the  end  of  the  field 
name.   Thus,  the  message  data  column  appeared  as  a  distinct 
column  with  a  uniform  margin. 
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d.   Columnar/Run-On  Format 

This  format  had  the  field  names  in  a  column  as 
above,  but  the  space  or  spaces  for  message  data  appeared 
without  spacing  to  the  immediate  right  of  the  field  name. 
Instead  of  a  single  column  of  field  names-message  data  as 
used  above,  three  columns  were  used  where  space  on  the 
display  area  existed;  otherwise,  two  columns  were  used. 
2 .   Message  Types 

An  independent  variable  of  secondary  interest  was 
message  type.   As  previously  discussed,  the  five  types 
selected  for  this  experiment  were  assumed  to  be  representa- 
tive of  the  total  population  of  messages  to  be  found  in  a 
fielded  system.   Data  pertinent  to  these  message  types  is 
shown  in  Table  II.   The  actual  messages  used  form  Appen- 
dix B. 

TABLE  II 
DATA  ON  MESSAGE  TYPES 


Msg.    Msg.          ,,    „  Number  of  Number  of 

m      *t            Msg.  Name  _,     ,  „ .  n  -, 

Type    Name           3  Characters  Fields 

1  CHKFIM   Check  Fire  Command         62  6 

2  EOMBDM   End  of  Mission  Battle       96  10 

Damage  Assessment 

3  TGTLST   Target  List  Input          151  14 

4  CFFTGR   Call  for  Fire  by           266  23 

Target  Number 

5  FORDRM   Fire  Order  Universal       573  54 
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3.  Subjects 

The  other  independent  variable  of  secondary  interest 
was  the  subjects  participating  in  the  test.   As  discussed, 
these  subjects  were  assumed  to  be  representative  of  the 
total  population  of  potential  system  operators.   However, 
their  previous  training  and  experience  might  have  affected 
their  performance  and  might  have  introduced  bias  into  the 
experiment.   The  inclusion  of  subjects  as  treatments  per- 
mitted an  analysis  of  their  effects  on  the  dependent  vari- 
ables to  be  made.   Eight  subjects  were  used. 

4 .  Replications 

The  independent  variable  of  tertiary  interest  was 
replication.   The  inclusion  of  this  variable  was  principally 
to  provide  a  check  to  see  if  the  experiment  was  conducted 
consistently. 

C   CONTROLLED  VARIABLES 

There  were  several  nuisance  variables  which  had  the 
potential  to  bias  the  outcome  of  the  experiment.   Subjects 
were  such  a  potential  source  of  bias  and  their  effects  were 
controlled  by  including  them  as  a  treatment  in  the  experi- 
mental design.   Other  nuisance  variables  could  not  be  exper- 
imentally controlled  as  were  subjects  but  had  to  be  con- 
trolled by  other  means. 

1 .   Abbreviations  and  Acronyms 

This  source  of  bias  was  controlled  both  experimentally 
and  statistically.   The  use  of  abbreviations  and  acronyms  in 
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all  message  types  is  not  necessarily  proportional  to  the 
length  of  the  message;  therefore,  the  normalization  of  the 
dependent  variables  as  discussed  in  Para.  II. A  was  also 
utilized  to  statistically  control  this  effect  by  keeping 
the  variation  more  nearly  constant  over  the  normalized  data 
base.   Standard  abbreviations  and  acronyms  were  used  for 
all  message  types  for  all  formats.   This  assumption  held 
these  effects  constant. 

2.  Experimenter  Differences 

Two  experimental  controls  were  used  to  minimize 
effects  due  to  potential  experimenter  differences.   First, 
all  experimenters  were  trained  to  administer  the  experiment 
in  a  standard  fashion  thus  minimizing  effects  due  to  differ- 
ent testing  procedures.   Second,  each  experimenter  presented 
each  format  twice  and  administered  no  more  than  one  test 
run  to  any  given  subject. 

3 .  Subject  Learning  Effects 

To  minimize  learning  effects,  two  experimental  con- 
trols were  used:   training  and  randomization  of  presentation 
of  formats.   All  subjects  completed  the  Marine  Tactical 
Command  and  Control  System  (MTACCS)  Test  Bed  Operations 
Training  Course  prior  to  being  administered  the  test.   Such 
training  ensured  familiarity  with  all  five  message  types 
and  all  four  formats.   Thus,  when  proceeding  from  a  test 
run  with  one  format  to  the  next  run  with  another,  a  subject's 
learning  effects  were  minimized.   Remaining  effects  which 
could  possibly  introduce  bias  were  reduced  by  randomizing 
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the  order  of  presentation  of  the  formats  and  message  types 
to  the  subjects. 

4.   Stimulus  Presentation 

a.  Message  Input  Stimuli 

The  stimulus  messages  were  presented  in  hard 
copy  form  only.   Radio  or  voice  presentation  was  deemed  too 
difficult  to  control  with  the  facilities  available  for  the 
test.   See  Appendix  B. 

b.  Format  Stimuli 

The  formats  used  for  this  test  were  presented 
to  the  subject  on  the  grid  display  screen.   One  format  was 
presented  at  a  time  in  the  order  shown  in  Table  III.   The 
random  ordering  of  the  presentation  of  each  format  minimized 
any  possible  effects  from  subject  learning  that  could  have 
occurred  if  the  formats  had  been  presented  to  all  the  sub- 
jects in  the  same  order. 

D.   STATISTICAL  DESIGN 

Since  it  was  desired  to  secure  data  from  every  combin- 
ation of  the  independent  variables,  a  randomized  complete 
block  factorial  design,  4  x5  ><8  with  two  replications,  was 
selected  for  this  experiment.   This  design  required  320 
data  points  with  which  to  test  the  hypotheses  and  provided 
an  extremely  high  power  for  the  test.   Table  III  presents 
the  test  schedule  required  to  support  this  design. 
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Run 

1 


Run 
2 


Run 
3 


TABLE  III 
TEST  SCHEDULE 


Sl   S2   S-^   S4   S5   S6   S7   S8 

FTFTFTFTFTFTFTFT 


41511451 

32144142 

23115325 

35223424 

2441133 
4^2355532 

11444253 

23252113 

15331514 

54522331 

E3      E4      El      E2      E3      E4      El      E2 

14445522 
55145243 
15453455 
33254115 

3  1  1449? 

2J31513^1^4^1^2 

41512251 

44324533 


E4      E3      E2      El      E4      E3      E2      El 


43351241 
25522331 
44231425 
35413512 
5  2  4  4  2  2  3  4 
3X4343543 
51154153 
22524152 
13235324 


El      E2      E3      E4      El      E2      E3      E4 
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TABLE    III       (Cont.) 


Sl      S2      S3      S4      S5      S6      S7      S8 

FTFTFTFTFTFTFTFT 


53155432 
34221225 
45523133 
24533551 

Run      4n34^3^3?2^2^1^1^ 
15251354 


4 


23342514 
11114441 
42412213 
51434142 

E2   El   E4   E3   E2   El   E4   E3 


1.   Assumptions 
a.   Model 

The  mathematical  model  assumed  for  this  design 
was  a  fixed  factor  model  as  developed  in  Ostle  [Ref .  1] : 

X.  .,  ,  =  U+R.+A.+B,  +C,  +  (AB)  .,  +  (AC)  .,+  (BC),  , 
ljkl       l   3   k   1      jk      jl      kl 

+  (ABC)  .,  ,+e  •  .,  , 
jkl   l^kl 

where    X.  ..  ,  =  observation  ijkl 
ljkl  J 

u  =  mean  effect  (constant) 

R.  =  effect  of  replicate  i,  i=l,2 

A.  =  effect  of  message  format  j ,  j=l,2,3,4 

B,  =  effect  of  message  type  k,  k=l,2,3f4/5 
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C,     =  effect  of  subject  1,  1=1,2,... ,8 

(AB)  .,  etc.  are  interaction  terms 

e.  .,  ,  =  random  error  for  observation  ijkl 
ljkl  J 

It  was  assumed  that  e.  .,  ,  was  distributed  normally  with  mean 

1  j  X.  x 

2 
zero  and  variance  a  where  e  is  homoschedastic  for  all  i,j, 

k,  and  1.   It  was  further  assumed  that  there  were  no  repli- 
cation by  treatment  interactions  and  that  the  observations 
were  independently  distributed  multivariate  normal. 

b.  Treatment  Effects 

As  discussed,  all  factors  in  this  model  were 
assumed  to  be  fixed.   There  is  no  doubt  of  this  for  the 
formats  and  message  types;  however,  the  subject  sample 
might  be  argued  to  be  random  rather  than  fixed.   Consider- 
ing the  experience  gained  by  all  the  subjects  through 
several  months  familiarization  with  the  test  bed  and  its 
equipment  and  their  overall  level  of  past  Marine  Corps 
operational  experience  and  training,  it  was  decided  that 
generalization  of  the  conclusions  made  from  this  test  over 
the  entire  population  of  possible  field  operators  would  not 
be  totally  valid.   Therefore,  of  necessity,  this  factor  had 
to  be  considered  fixed,  also. 

c.  Effects  of  Nonconformity  to  Assumptions 

In  general,  the  consequences  are  not  serious 
when  the  assumptions  made  in  connection  with  the  analysis 
of  variance  are  not  strictly  satisfied.   For  example,  minor 
deviations  from  normality  and/or  some  degree  of  heterosche- 
dasticity  will  have  little  effect  on  the  usual  tests  and 
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the  resulting  inferences.   Reference  2  discusses  this  aspect 
of  experimental  design  thoroughly. 
2 .   Sample  Size 

To  determine  the  proper  sample  size  to  use  for  this 
test,  one  has  to  use  ideas  developed  by  Tang  [3]  and  used 
by  Pearson  and  Hartley  [4]  in  the  development  of  their 
power  function  curves.   Using  these  developments,  one  cal- 
culates the  sample  size  required  to  yield  a  specified  power 
for  the  desired  test  utilizing  this  equation: 


E.D.2 
1  1 


*  =  -^— * — 

o//n~ 

where    <\>      =   power  function 

D.  =  difference  between  the  effect  of  the  i 

treatment  and  the  mean  effect  of  all  treatments 

k   =  number  of  treatments  (i.e.,  number  of  levels 
of  the  treatment  in  question 

a   =  standard  deviation  of  errors 

n   =  number  of  observations  required  for  each 
treatment  level 

Since,  for  this  experiment,  a  and  D.  were  not  known,  a  modi- 
fication to  this  equation  was  made  to  circumvent  these 
problems.   It  was  assumed  that  a  maximum  detectable  treat- 
ment difference  of  one  a  about  the  mean  would  be  sufficient 

2 
for  this  experiment.   It  can  be  shown  that  the  minimum  E.D. 

occurs  when  two  of  the  treatment  differences  are  -  —  and 

2 

—  and  the  remaining  differences  are  0  [Ref .  7] .   Then 
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Vi  =  (-t)2  +  (i)2  +  °  +  --0  =  4 

Hence,  if  the  maximum  detectable  difference  is  other  than 
that  chosen,  the  power  of  the  test  will  increase.   Now,  the 
above  equation  for  <J>  becomes 


|02 


,    \2k      /n 

*  =  ^7=   =^2k 
o/vn 


This  form  was  much  simpler  to  utilize  and  was  no  longer 
sensitive  to  errors  in  estimating  a.   Using  this  power 
function  with  the  proper  degrees  of  freedom  and  a  =  0.05, 
the  available  power  can  be  determined  for  various  sample 
sizes  from  the  curves  produced  by  Pearson  and  Hartley  [4] . 
As  shown  in  Table  IV,  a  power  of  approximately  0.90  was 
attainable  for  the  test  of  format  differences  with  as  few 
as  three  subjects,  each  subject  using  two  samples  of  each 
message  type  for  each  format;  whereas,  four  subjects  were 
required  to  get  a  similar  power  for  the  test  of  message 
types.   Since  there  were  40  observations  for  each  subject, 
effects  of  subject  differences  could  be  tested  with 
extremely  high  power  for  almost  any  number  of  subjects. 
However,  as  the  use  of  eight  subjects  did  not  add  mater- 
ially to  the  cost  or  time  required  but  did  enhance  the 
power  of  any  test  desired,  eight  subjects  were  used. 
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TABLE  IV 
AVAILABLE  POWERS  FOR  THE  TEST 


No.  of 
Subj  . 

Obs./Subj ./ 
Cell 

Degrees  of 
Freedom  for 
Error  Term 

Power 

Format 

Msg. 
Type 

12 

2 

239 

0.999 

0.999 

10 

2 

199 

0.999 

0.999 

8 

2 

159 

0.999 

0.999 

6 

2 

119 

0.999 

0.982 

4 

2 

79 

0.972 

0.890 

4 

3 

158 

0.999 

0.984 

3 

2 

59 

0.  895 

0.770 

3 

3 

118 

0.986 

0.950 

2 

2 

39 

0.710 

0.550 

2 

3 

78 

0.910 

0.790 

2 

4 

117 

0.968 

0.900 
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III.   LOGISTICS  AND  TRAINING 

A.  SUBJECTS 

The  eight  subjects  used  for  this  test  were  Marine  Corps 
officer  and  enlisted  personnel  assigned  to  the  MTACCS  Test 
Bed  which  has  direct  responsibility  for  developing  MIFASS. 
Names  of  22  eligible  officers  and  men  formed  a  pool  from 
which  eight  were  drawn  randomly  to  act  as  subjects.   Pre- 
requisites for  eligibility  were  previous  training  or 
experience  in  fire  support  coordination,  artillery  coordi- 
nation/ target  data  analysis,  and  test  bed  operations.   A 
capability  of  an  extremely  high  or  low  typing  speed  rendered 
potential  subjects  ineligible  as  it  was  desired  to  have 
typing  speed  on  a  level  representative  of  personnel  who  will 
serve  in  a  fielded  MIFASS.   Each  subject  had  completed  the 
MTACCS  Test  Bed  Operations  Course  which  covered  those 
subjects  listed  in  Table  V.   Prior  to  their  first  test  run, 
the  subjects  were  thoroughly  briefed  on  the  experiment. 

B.  EXPERIMENTERS 

The  four  experimenters  necessary  for  this  test  were 
selected  similarly  to  the  subjects  except  that  all  test  bed 
personnel,  including  the  civilian  contractor  employees, 
were  considered  eligible  for  duty  as  experimenters.   Avail- 
ability was  the  prime  consideration  in  experimenter  selec- 
tion.  Those  selected  were  familiar  with  MTACCS  Test  Bed 
operations  through  the  above  mentioned  course  and  were  all 
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TABLE  V 

SUBJECTS  COVERED  IN  THE  MTACCS 
TEST  BED  OPERATIONS  COURSE 


MTACCS  Concept 

Role  of  the  Test  Bed 

MIFASS  TSM  Concept 

Concept  of  Test  Bed  Operations 

Display  and  Entry  Devices  (I/O) 
(Included  practical  exercises) 


Marine  Corps  personnel.   Additional  training  included 
experiment  familiarization  sessions  conducted  by  the  test 
administrator,  practice  sessions,  and  a  pilot  study  as 
discussed  in  Para.  IV. A.   Those  selected  also  attended  the 
subject  briefings. 

C.   HARDWARE  AND  SOFTWARE  REQUIREMENTS 

1.  Hardware 

The  hardware  listed  in  Table  VI  was  utilized  for 
the  performance  of  this  experiment. 

2 .  Software 

The  software  used  for  this  experiment  is  discussed 
thoroughly  in  Ref.  5.   Since  it  is  not  in  the  public  domain 
nor  pertinent  to  a  general  discussion  of  the  test,  a  listing 
is  omitted  here. 
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TABLE  VI 
HARDWARE  REQUIRED  FOR  THIS  EXPERIMENT 


Unit 


Model 

Quantity 

3300 

1 

243-1 

1 

600 

2 

TR-1720 

1 

TC-201 

1 

CDC  Computer  System 
CDC  Grid  Display  System 
Call  Director 
Voice  Recorder 
Time  Code  Generator 


D.   TEST  EQUIPMENT  AND  MATERIAL 

1 .  Subject 

In  order  for  the  subjects  to  function  efficiently, 
they  had  at  their  disposal  on  the  grid  display  console  a 
reference  chart  and  a  MTACCS  Glossary.   The  reference  chart 
contained  field  header  data  and  allowable  field  data  entries 
used  in  all  MIFASS  messages  and  displays.   The  MTACCS 
Glossary  contained  all  the  abbreviations  and  acronyms  to  be 
used. 

2 .  Experimenter 

Each  experimenter  had  at  his  disposal  the  following 
items:   a  stopwatch.  Parametric  Data  Collection  Forms, 
Subject  Reaction  Forms,  the  voice  recorder,  and  the  input 
message  stimuli. 
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E.  STIMULUS  MATERIAL 

All  stimulus  materials  were  developed  by  Hughes  Aircraft 
Company  in  conjunction  with  Test  Bed  officers.   Message 
input  stimuli  for  the  five  messages  used  appear  in  App.  B. 
The  formats  presented  to  the  subjects  for  each  message 
appear  in  App.  A. 

F.  DATA  COLLECTION  FORMS 

To  support  the  analyses  necessary  for  this  test,  two 
data  collection  forms  were  necessary. 

1.  Parametric  Data  Collection  Form 

As  depicted  in  Fig.  2,  this  form  was  used  to  collect 
data  for  use  in  determining  speed  and  accuracy  measures. 
One  form  was  completed  by  the  applicable  experimenter  for 
each  subject  for  each  format.   A  total  of  32  forms  were 
required. 

2 .  Subject  Reaction  Form 

As  discussed  earlier,  this  form  was  necessary  as  an 
intermediate  step  in  the  nonparametric  analysis.   A  total  of 
32  forms  were  required  as  each  subject  completed  one  for 
each  format  immediately  upon  completion  of  the  run  testing 
that  format.   Figure  1  depicts  this  form. 
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Subject 
Format 


Date 


Experimenter 


Message  Type    Time    No .  of  Errors 


Remarks 


Experimenter  Notes 


Figure  2.   Parametric  Data  Collection  Form, 
This  form  was  used  to  gather 
speed  and  accuracy  data  for  the 
ANOV. 
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IV.   TEST  PROCEDURE 

A.   PILOT  STUDY 

1.  Purpose 

The  pilot  study  was  primarily  conducted  in  order  to 
validate  the  test  design,  determine  if  the  desired  data 
could  be  collected,  and  see  if  the  required  analyses  could 
be  made  using  this  data.   Secondary  considerations  were 
training  of  experimenters,  refinement  of  data  collection 
procedures  and  forms,  and  detection  and  correction  of  errors 
in  the  test  stimuli. 

2 .  Administration 

a.  Materials 

The  test  stimuli,  equipment,  and  data  collection 
forms  used  were  those  to  be  used  during  the  actual  test. 

b.  Procedure 

The  test  administrator  selected  those  indivi- 
duals to  be  used  as  experimenters  during  the  test.   Trial 
runs  were  made  with  each  selectee  acting  as  the  subject  to 
familiarize  him  with  the  entire  test.   Each  selectee  then 
performed  as  an  experimenter  for  other  runs  using  another 
experimenter  as  a  subject.   Data  was  collected  as  it  was  to 
be  collected  during  the  actual  test.   This  procedure  was 
followed  until  the  test  administrator  felt  that  the  level 
of  experience  was  great  enough  to  minimize  any  effects  due 
to  experimenter  differences. 
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c.   Results 

A  trial  analysis  was  made  using  the  data 
gathered  during  the  pilot  study.   The  test  design  was 
confirmed  as  was  the  suitability  of  the  data  collection 
forms . 

B.   TEST  PROCEDURE 

The  actual  test  runs  were  conducted  as  follows: 

Prior  to  each  run,  the  test  administrator  ensured 
that  the  system  was  operating  properly  and  that  the  correct 
software  was  loaded. 

The  experimenter  presented  the  subject  with  the 
proper  message  stimuli  and  recorded  the  event  on  the  voice 
recorder. 

The  format  to  be  filled  in  by  the  subject  was  called 
to  the  screen  from  the  memory. 

When  the  format  to  be  tested  appeared,  the  experi- 
menter recorded  the  event  and  started  his  stopwatch. 

The  subject  entered  the  necessary  data  using  the 
alphanumeric  keyboard  on  the  grid  console. 

Upon  completion  of  his  task,  the  subject  pressed 
the  STORE  RECORD  key.   The  experimenter  stopped  his  stop- 
watch and  recorded  the  time  on  the  voice  recorder  and  the 
Parametric  Data  Collection  Form. 

The  above  steps  were  repeated  until  that  run  was 
complete.   The  subject  then  completed  his  Subject  Reaction 
Form. 


37 


C.   SUBJECT  BRIEFING 

Prior  to  commencement  of  the  first  test  run,  a  thorough 
briefing  was  given  to  all  subjects  and  experimenters.   The 
briefing  served  to  acquaint  the  subjects  with  the  test,  its 
purpose,  and  its  procedures.   The  test  administrator  pre- 
viewed the  test,  answered  any  questions  the  subjects  or 
experimenters  had,  and  provided  each  subject. with  the  test 
schedule.   The  subjects  were  cautioned  not  to  discuss  their 
reactions  with  other  subjects  until  after  completion  of  the 
entire  test. 
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V.   ANALYSIS 

A.   PARAMETRIC  ANALYSIS  OF  VARIANCE  (ANOV) 
1.   Procedure 

a.  Data  Collection 

Test  data  for  the  dependent  variables  speed  and 
accuracy  were  collected  on  the  Parametric  Data  Collection 
Form.   The  time  required  to  enter  each  message  stimulus  into 
each  format  by  each  subject  was  recorded  by  the  experimenter 
but  the  accuracy  observations  were  collected  differently. 
Errors  were  detected  through  the  use  of  a  compare  program 
by  which  the  computer  compared  actual  entries  with  the  cor- 
rect entries  and  provided  a  printout  of  all  entries  con- 
taining errors.   The  Test  Administrator  then  manually 
counted  and  recorded  these  errors.   The  data  collected  for 
both  parametric  analyses  is  presented  in  App.  C. 

b.  Normalization  of  Data 

In  order  to  preclude  effects  due  to  the  differ- 
ing lengths  of  the  message  types,  the  test  data  had  to  be 
normalized. 

(1)   Speed.   The  normalizing  factors  for  speed 
of  data  entry  were  the  reciprocals  of  the  number  of  Keying 
actions  necessary  to  enter  a  message  type  into  a  particular 
format.   For  example,  for  Format  1  Message  Type  1,  the 
number  of  keying  actions  required  was  26,  so  the  time  for 
each  subject  to  enter  the  data  for  this  combination  was 
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divided  by  26  to  obtain  the  normalized  times.   These  times 
appear  in  App.  D. 

(2)   Accuracy.   The  accuracy  scores  were  norm- 
alized in  a  similar  manner;  however,  the  normalizing  factors 
were  the  reciprocals  of  the  number  of  complete  data  elements 
to  be  entered  in  each  format  for  each  message  type.   These 
normalized  observations  appear  in  App.  D. 

c.  Organization  of  Data 

In  order  to  minimize  the  effort  necessary  for 
performing  the  ANOV  for  both  speed  and  accuracy,  the  data 
in  App.  D  was  rearranged  into  interaction  tables  as  shown 
in  App.  E   [1]  . 

d.  Computations  and  ANOV  Tables 

The  computations  necessary  for  obtaining  the 
results  to  be  entered  in  the  ANOV  tables  were  accomplished 
using  the  set  of  equations  summarized  in  Ref.  1.   The 
general  form  of  an  ANOV  table  is  discussed  in  Ref.  1,  also; 
therefore,  it  will  not  be  presented  here. 
2 .   Results 

a.   ANOV  for  Speed  of  Data  Entry 

The  ANOV  table  obtained  for  the  dependent  vari- 
able speed  in  this  test  is  given  in  Table  VII.  The  effects 
of  the  different  format  levels  and  message  types  are  sig- 
nificant at  a  =  0.05  as  are  the  interactions  of  format  with 
both  message  type  and  subject.  The  acceptance  or  rejection 
of  the  hypotheses  concerning  speed  of  data  entry  is  given 
below: 
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Rejected 

H  0 
o3 

Accepted 

o5 

Rejected 

H   -7 

o7 

Rejected 

H09 

Rejected 

H011 

Accepted 

Hol3 

Accepted 

b.   ANOV  for  Accuracy  of  Data  Entry 

Results  obtained  from  the  ANOV  of  the  dependent 
variable  accuracy  are  displayed  in  Table  VIII.   Significant 
effects  are  exhibited  by  formats,  message  types,  and  their 
interaction  at  a  =  0.05. 

During  the  test,  faulty  design  in  the  program 
for  Message  Type  2  entered  in  Format  1  required  a  data 
element  to  be  carried  over  from  one  line  to  the  next.   The 
margin  for  the  80    character  on  the  grid  screen  was  not 
visible  so  several  subjects  had  difficulty  determining 
when  to  cease  entering  data  on  the  first  line  and  to  con- 
tinue on  the  second.   The  higher  incidence  of  errors  for 
Format  1  and  Message  Type  2  made  an  adjusted  ANOV  for 
accuracy  desirable.   Therefore,  another  ANOV  was  conducted 
with  the  errors  which  were  directly  attributable  to  this 
programming  problem  omitted  from  the  observations.   Table  IX 
displays  the  adjusted  ANOV  results.   As  shown,  format  and 
message  type  effects  are  no  longer  significant  though 
their  interaction  remains  so.   For  the  two  ANOV  for  accuracy 
of  data  entry,  the  acceptance  or  rejection  of  the  hypotheses 

is: 
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TABLE  VII 
ANOV  TABLE  FOR  SPEED  OF  DATA  ENTRY 


Source 

df 

SS 

MS 

F 

Replications 

1 

.096 

.096 

1.215 

Treatments 

Format  (A) 

3 

2.330 

.777 

9.835* 

Msg.  Type  (B) 

4 

.663 

.166 

2.101 

Subject  (C) 

7 

8.706 

1.244 

15.746* 

AB 

12 

2.552 

.213 

2.696* 

AC 

21 

3.311 

.158 

2.000* 

BC 

28 

1.871 

.067 

.848 

ABC 

84 

5.798 

.069 

.873 

Error 

159 

12.613 

.079 

Total 

319 

37.938 

Significant  at  a  =  0.05.   All  others  were  tested  against 
a  =  0.05  and  were  not  significant. 


42 


TABLE  VIII 
ANOV  TABLE  FOR  ACCURACY  OF  DATA  ENTRY 


Source 

df 

SS 

MS 

F 

Replications 

1 

.00006 

.00006 

t 

.03 

Tr  atments 

Formats 

(A) 

3 

.02679 

.00893 

5. 

,02* 

Msg.  Typ< 

2S  (B) 

4 

.03520 

.00880 

4. 

,94* 

Subjects 

(C) 

7 

.01167 

.00170 

i 

,96 

AB 

12 

.13266 

.01106 

6. 

,21* 

AC 

21 

.03989 

.00190 

1. 

,07 

BC 

28 

.05343 

.00191 

1. 

,07 

ABC 

84 

.11638 

.00139 

■ 

,78 

Error 

159 

.28319 

.00178 

Total 

319 

.69927 

* 

Significant  at  a  =  0.05.   All  others  were  tested  against 

a  =  0.05  and  were  not  significant. 
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TABLE  IX 

ADJUSTED  ANOV  TABLE  FOR  ACCURACY 
OF  DATA  ENTRY 


Source 

df 

SS 

MS 

F 

Replications 

1 

.00026 

.00026 

.17 

Treatments 

Formats 

(A) 

3 

.01105 

.00368 

2.44 

Msg.  Typ< 

BS  (B) 

4 

.00824 

.00206 

1.36 

Subjects 

(C) 

7 

.01844 

.00264 

1.75 

AB 

12 

.11210 

.00934 

6.19* 

AC 

21 

.03085 

.00147 

.97 

BC 

28 

.05538 

.00198 

1.31 

ABC 

84 

.11288 

.00134 

.89 

Error 

159 

.24037 

.00151 

Total 

319 

.58957 

Significant  at  a  =  0.05.   All  others  were  tested  against 
a  =  0.05  and  were  not  significant. 


44 


H 


o2 


H 


o4 


H 


06 


H 


08 


H 


olO 


H 


ol2 


H 


ol4 


Original 
Rejected 
Rejected 
Accepted 
Rejected 
Accepted 
Accepted 
Accepted 


Adjusted 
Accepted 
Accepted 
Accepted 
Rejected 
Accepted 
Accepted 
Accepted 


B.   NONPARAMETRIC  ANALYSIS 
1.   Procedure 

a.  Data  Collection 

Immediately  upon  completion  of  each  test  run 
each  subject  completed  a  Subject  Reaction  Form  on  which  he 
marked  his  evaluation  of  the  ease  of  use  and  suitability 
of  the  format  tested. 

b.  Analysis 

The  Friedman  Test  is  discussed  thoroughly  by 
Conover  and  Kirk  [Refs.  6  and  7].   It  requires  two  assump- 
tions (which  are  considered  valid  for  this  test) :   (1)  each 
subject's  evaluation  of  the  four  formats  will  be  independent 
of  every  other  subject's  evaluation,  and  (2)  a  subject's 
evaluations  may  be  ranked  in  some  manner.   The  analysis  for 
both  ease  of  use  and  suitability  is  given  in  App.  F. 
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2.   Results 

a.  Ease  of  Use 

The  test  statistic  for  this  analysis  was  15.96 
which,  when  compared  with  the  0.9  5  quantile  of  a  Chi  Square 
random  variable  with  three  degrees  of  freedom,  is 
significant. 

b.  Suitability 

For  this  analysis,  the  test  statistic  was  11.64 
which  is  also  significant  at  the  0.05  level. 

c.  Hypotheses 

Both  H  ,  _  and  H  ,  ,.  were  rejected  at  the  0.05 
ol5       ol6         J 

level,  i.e.,  the  subjects  were  not  indifferent  to  the 
formats  as  hypothesized. 
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VI.   CONCLUSION 

A.   PARAMETRIC  ANOV 

1.   Speed  of  Data  Entry 

The  results  of  the  ANOV  for  this  dependent  variable 
clearly  show  that  format  variation  does  affect  speed  of  data 
entry;  thus,  the  primary  purpose  of  this  test  was  success- 
fully accomplished.   However,  due  to  significant  inter- 
actions of  the  formats  with  both  message  types  and  subjects, 
interpretation  of  significant  main  effects  must  be  qualified. 
Figure  3  illustrates  the  type  of  interaction  detected  between 
formats  and  subjects. 


13- 

12- 

11- 

Speed   10. 

9' 
8" 


Subject  4 
Subject  2 


j_ 


_L 


2  3 

Format 


Figure  3.   Example  of  a  Typical  Format-Subject  Interaction 

It  appeared  that  groups  of  subjects  reacted  in  similar  man- 
ners but  that  each  group  reacted  differently  from  the  other 
groups.   For  example,  subjects  1,2,7,  and  8  all  reacted  to 
the  formats  in  the  same  order,  i.e.,  Formats  4,3,1,  and  2 
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(from  fastest  to  slowest),  whereas,  Subjects  4,5,  and  6 
reacted  in  the  order  3,4,2,1,  and  for  Subject  3  the  order 
was  2,3,1,4.   No  discernible  pattern  of  subject  character- 
istics such  as  I.Q.,  typing  speed,  test  bed  experience,  MOS , 
etc.,  was  evident  within  the  groups.   Though  the  subject 
levels  were  considered  fixed,  it  may  be  concluded  that  no 
single  format  will  be  best  for  all  the  operators  in  a  fielded 
system.   For  any  format  chosen,  under  the  present  concept, 
some  operators  will  not  be  able  to  function  optimally.   This 
implies  that  rigid  selection  and  training  programs  must  be 
instituted  in  order  to  minimize  the  effects  due  to  operator 
variations.   The  format-message  type  interaction  implies 
that  some  message  types  are  inherently  faster  (or  slower) 
when  entered  in  certain  formats. 
2 .   Accuracy  of  Data  Entry 

The  original  ANOV  clearly  rejected  the  hypothesis 
that  neither  format  nor  message  type  would  affect  accuracy 
of  data  entry.   However,  when  ignoring  those  errors  directly 
attributable  to  the  program  error,  an  adjusted  ANOV  resulted 
in  the  acceptance  of  these  hypotheses.   The  results  of  the 
adjusted  ANOV  indicate  that  these  factors  have  little  effect 
on  accuracy  of  data  entry;  however,  their  interaction  is 
significant.   This  implies  that  some  of  the  message  types 
are  inherently  more  accurate  (or  inaccurate)  when  used  with 
certain  formats. 
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B.  NONPARAMETRIC  ANALYSIS 

Both  hypotheses  tested  were  rejected.   The  subjects 
tended  to  consistently  rank  the  formats  so  that  one  or  more 
were  favored  over  the  others.   Viewing  the  totaled  ranks  for 
each  format  one  may  infer  that  the  subjects  tended  to  favor 
Format  3  over  the  others  for  both  categories.   In  both  cases 
the  overall  ranking  of  the  formats  was  3,4,1,  and  2. 

C.  SUMMARY 

1.  Purpose  of  the  Test 

As  discussed  in  Para.  I. D, this  test  was  designed  to 
evaluate  differences  in  operator  data  entry  speed  and 
accuracy  resulting  from  the  use  of  alternative  message  for- 
mats when  used  with  differing  types  of  I/O  data.   The  test 
was  not  designed  to  determine  which  format  would  be  best  but 
to  determine  whether  or  not  different  formats  would  affect 
the  dependent  variables.   To  this  extent,  the  purposes  of 
this  test  were  accomplished.   Different  formats  do  affect 
the  speed  of  data  entry  but  do  not  affect  accuracy 
appreciably. 

2 .  Inferences 

Due  to  the  format-subject  interaction,  no  one  format 
can  be  inferred  with  certainty  to  be  superior  to  the  others. 
However,  the  mean  speed  for  Format  3  was  the  smallest  of 
the  four  formats  tested  and  its  error  rate  was  very  low. 
It  received  no  rank  below  two  for  ease  of  use  and  was  con- 
sidered first  by  four  subjects  and  tied  for  first  by  one. 
Concerning  operational  suitability,  Format  3  was  ranked 
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first  by  four  subjects  and  tied  for  first  by  one.   It  was 
definitely  preferred  over  the  other  three  formats  tested  by 
the  majority  of  the  subjects  even  though  it  was  not  fastest 
for  several.   Thus,  Format  3  may  be  inferred,  with  reserva- 
tions, to  be  superior  for  use  in  a  fielded  system. 

D.   RECOMMENDATIONS 

As  shown,  the  evidence  tended  to  place  Format  3  over 
the  other  three  formats  for  use  in  the  MIFASS.   Further 
testing  should  be  contemplated  to  determine  the  reasons 
for  subject  and  message  type  effects.   If  subject  effects 
can  be  overcome  through  selection  and  training  of  operators, 
then  further  development  can  be  directed  toward  determining 
optimum  formats  for  the  message  types  to  be  used.   This 
study  can  be  integrated  into  the  ongoing  development  of  the 
entire  system  if  time  does  not  permit  other  tests  to  be 
conducted  alone  and  proper  precautions  are  taken  to  prevent 
confounding  of  variables.   Another  aspect  to  be  approached 
is  the  weighting  of  message  types  with  regard  to  their 
importance  to  the  success  of  the  system.   If  a  particular 
message  type  is  crucial  to  the  success  of  the  system,  it 
should  have  more  weight  when  optimal  formats  are  being 
considered. 
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APPENDIX  A: 


FORMATS  USED  IN  THIS  TEST 
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APPENDIX  B:   MESSAGE  INPUT  STIMULI 


MESSAGE  TYPE  1 


Field  Name 

Abbrevi 

Check  Fire  Msg. 

CHKFIM; 

From 

FR: 

To 

TO: 

Target  Number 

TGT  NO: 

Request  Number 

RQNO: 

Date  Time  Group 

DTG: 

Requested  Entry 


AR61 

BC30 

BD2121 

162310 

XXXXXXXX 


MESSAGE  TYPE  2 


Field  Name 

Abbreviation 

Requested  Entry 

End  of  Mission 

EOMBDM; 

Battle  Damage 

From 

FR: 

AA31 

To 

TO: 

BA20 

Target  Number 

TGT  NO: 

BB0100 

Request  Number 

RQNO: 

210000 

Casualties 

CAS: 

00025 

%  Destroyed 

%DEST: 

99 

Enemy  Posture 

ENMY  PSTR: 

Burn 

Date  Time  Group 

DTG: 

XXXXXXXX 
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MESSAGE  TYPE  3 


Field  Name 

Abbrevi 

Target  List  Input 

TGTLST ; 

From 

FR: 

To 

TO: 

UTM  Location 

GRID: 

Altitude 

ALT: 

Target  Type/Subtype 

TTYP: 

Size 

SIZE: 

Attitude 

ATT: 

Degree  of  Protection 

DOP: 

Quantity 

QTY: 

Target  Priority 

PRI: 

Target  Status 

STATUS : 

Date  Time  Group 

DTG: 

Requested  Entry 


CA35 

AC40 

DE64022610 

00045 

CEN    /BN 

100X100 

08 

HV 

1 

1 

CONF 

XXXXXXXX 
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MESSAGE  TYPE  4 


Field  Name 


Abbreviation 


Requested  Entry 


Call  for  Fire  by 

CFFTGR; 

Target  Number 

From 

FR: 

To 

TO: 

Target  Number 

TGT  NO: 

Target  Type/Subtype 

TTYP: 

Quantity 

QTY: 

Size 

SIZE: 

Degree  of  Protection 

DOP: 

Target  Mobility 

TGT  MOV: 

Attitude 

ATT: 

Method  of  Engagement 

METH  ENG 

Weapon  Category 

WCAT: 

Weapon 

WPN: 

Ordnance 

ORD: 

Fuze 

FZ: 

Distribution 

DISTRB: 

Method  of  Fire 

MOF: 

Method  of  Control 

MOC: 

Observer  Target 

OT  DIR: 

Direction 

Request  Number 

RQNO: 

Date  Time  Group 

DTG: 

AG31 

AG20 

MT0732 

VEH    /AC FT 

5 

200X200 

MDM 

STAT 

10 

AREA 

AIR 

2000 

BMBLT 

Q 

LAT 

CONT 

FFE 

018 

012116 
XXXXXXXX 
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MESSAGE  TYPE  5 


Field  Name 


Abbreviation 


Requested  Entry 


Fire  Order  Universal  FORDRM; 

From  FR: 

To  TO: 

Weapon  Category  WCAT: 

Target  Number  TGT  NO: 

Request  Number  RQNO: 

UTM  Coordinates  GRID: 

Altitude  ALT: 

Method  of  Control  MOC: 

Method  of  Engagement  METH  ENG: 

Distribution  of  Fire  DISTRB: 

Time  to  Open  Fire  OPN  FIR: 

Adjusting  Unit  UADJ: 

Number  of  Adjusting  RDS : 
Rounds 

Adjusting  Ordnance  ORD: 

Adjusting  Fuze  FZ : 

Adjusting  Lot  LOT: 

Adjusting  Charge  CHG: 

Adjusting  Angle  ANGLE: 

Adjusting  Time  on  TOT: 
Target 

Fire  for  Effect  Unit  UFFE: 

FFE  No.  of  Rounds  RDS: 

FFE  Ordnance  ORD: 


EC10 

AEOO 

ARTY 

BR2361 

390061 

DE65022500 

00020 

ADJ 

MRK/HA 

ZN 

0600 

G/3/10 

00003 

HE 

Q 

XY 

4 

26 

0640 

H/3/10 

00004 

ICM 
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Field  Name 

Abbrev 

FFE  Fuze 

FZ: 

FFE  Lot 

LOT: 

FFE  Charge 

CHG: 

FFE  Angle 

ANGLE: 

FFE  Gun  Target 

Line 

GTAZ: 

FFE  Unit 

UFFE: 

FFE  No.  of  Rounds 

RDS: 

FFE  Ordnance 

ORD: 

FFE  Fuze 

FZ: 

FFE  Lot 

LOT: 

FFE  Charge 

CHG: 

FFE  Gun  Target 

Line 

GTAZ: 

FFE  Unit 

UFFE: 

FFE  No.  of  Rounds 

RDS: 

FFE  Ordnance 

ORD: 

FFE  Fuze 

FZ: 

FFE  Lot 

LOT: 

FFE  Charge 

CHG: 

FFE  Angle 

ANGLE: 

FFE  Gun  Target 

Line 

GTAZ: 

Date  Time  Group 

DTG: 

Requested  Entry 


TI 

TW 

3 

16 

3200 

1/3/10 

00006 

SMKRED 

TI 

YX 

5 

1800 

K/3/10 

00002 

WP 

VT 

TE 

2 

24 

4000 

XXXXXXXX 
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APPENDIX  E:   INTERACTION  TABLES 


SPEED  OF  DATA  ENTRY 


AXBXC 


C/B 

1 

2 

1 

2 

3 

4 

5 

1 

2 

3 

4 

5 

1 

1.308 

2.400 

1.758 

1.870 

1.723 

2.259 

2.079 

1.880 

2.449 

2.024 

2 

1.423 

2.000 

2.919 

2.039 

1.764 

3.963 

1.922 

1.956 

2.858 

2.262 

3 

1.808 

1.800 

3.077 

2.747 

2.231 

1.519 

1.743 

2.132 

2.398 

1.984 

4 

2.153 

2.915 

2.343 

2.600 

2.610 

2.259 

2.300 

2.403 

2.608 

2.373 

5 

2.616 

3.836 

2.799 

3.342 

2.940 

4.111 

2.124 

2.636 

3.245 

3.384 

6 

1.615 

2.171 

2.231 

2.727 

2.271 

2.888 

1.643 

1.752 

1.990 

2.226 

7 

2.077 

2.550 

2.403 

2.374 

2.371 

2.852 

2.802 

2.668 

3.038 

3.508 

8 

2.115 

3.150 

2.133 

2.381 

2.174 

2.555 

4.330 

2.766 

3.539 

3.140 

C/B 

3 

4 

1 

2 

3 

4 

5 

1 

2 

3 

4 

5 

1 

1.770 

1.436 

1.838 

2.310 

1.888 

1.461 

1.368 

1.542 

1.617 

2.049 

2 

1.500 

1.513 

2.370 

1.584 

1.808 

1.346 

1.242 

1.738 

1.713 

2.137 

3 

2.616 

1.436 

2.322 

1.925 

2.380 

3.269 

1.846 

2.694 

2.691 

2.109 

4 

1.923 

1.769 

2.387 

2.136 

2.084 

2.423 

2.103 

2.177 

2.363 

2.291 

5 

2.116 

2.411 

2.581 

2.523 

2.684 

2.423 

2.026 

3.242 

2.  397 

2.733 

6 

1.423 

1.436 

1.710 

1.  800 

1.930 

1.  808 

1.692 

2.064 

1.  894 

2.224 

7 

2.153 

2.410 

2.564 

2.881 

2.726 

2.192 

2.281 

2.156 

2.543 

2.319 

8 

2.730 

2.539 

2.565 

2.577 

2.763 

2.923 

2.229 

2.215 

2.713 

2.298 
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AXB 


B/A 

1 

2 

3 

4 

1 

15.115 

22.406 

16.231 

17.845 

2 

20.822 

18.943 

14.950 

14.787 

3 

19.663 

18.193 

18.337 

17.828 

4 

20.080 

22.125 

17.736 

17.931 

5 

18.084 

20.901 

18.263 

18.160 

AXC 


C/A 

1 

2 

3 

4 

1 

9.059 

10.691 

9.242 

8.037 

2 

10.145 

12.961 

8.775 

8.176 

3 

11.663 

9.776 

10.679 

12.609 

4 

12.621 

11.943 

10.299 

11.357 

5 

15.533 

15.500 

12.315 

12.821 

6 

11.015 

10.499 

8.299 

9.682 

7 

11.775 

14.868 

12.734 

11.491 

8 

11.953 

16.330 

13.174 

12.378 

BXC 


C/B 

1 

2 

3 

4 

5 

1 

6.798 

7.283 

7.018 

8.246 

7.684 

2 

8.232 

6.677 

8.983 

8.194 

7.971 

3 

9.212 

6.825 

10.225 

9.761 

8.704 

4 

8.758 

9.087 

9.310 

9.707 

9.358 

5 

11.266 

10.397 

11.258 

11.507 

11.741 

6 

7.734 

6.942 

7.757 

8.411 

8.651 

7 

9.274 

10.043 

9.791 

10.836 

10.924 

8 

10.323 

12.248 

9.679 

11.210 

10.375 
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AXBXC 


ACCURACY  OF  DATA  ENTRY 


C/B 

1 

2 

1 

2 

3 

4 

5 

1 

2 

3 

4 

5 

1 

.143 

.250 

.143 

.023 

2 

.286 

.083 

.048 

.143 

.048 

.023 

3 

.143 

.048 

.023 

.166 

.115 

4 

.143 

.083 

.023 

5 

.286 

.083 

.096 

.023 

.250 

.023 

6 

.286 

.083 

.048 

.048 

.023 

7 

.286 

.048 

.023 

8 

.286 

.023 

.143 

.249 

.023 

C/B 

4 

5 

1 

2 

3 

4 

5 

1 

2 

3 

4 

5 

1 

.250 

.048 

.143 

.023 

2 

.083 

.046 

3 

.250 

.083 

.048 

.023 

4 

5 

6 

.083 

.048 

.166 

.144 

7 

.048 

.023 

.023 

8 

.048 

. 
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AXB 


B/A 

1 

2 

3 

4 

1 

.500 

.500 

2 

1.859 

.429 

.143 

3 

.249 

.498 

.249 

.166 

4 

.288 

.096 

.240 

.144 

5 

.069 

.276 

.046 

.092 

AXC 


C/A 

1 

2 

3 

4 

1 

.143 

.416 

.298 

.166 

2 

.417 

.214 

.083 

.0^6 

3 

.214 

.281 

.404 

4 

.143 

.106 

5 

.488 

.273 

6 

.417 

.071 

.131 

.310 

7 

.334 

.023 

.071 

.023 

8 

.309 

.415 

.048 

BXC 


C/B 

1 

2 

3 

4 

5 

1 

.500 

.429 

.048 

.046 

2 

.429 

.166 

.096 

.069 

3 

.250 

.143 

.249 

.096 

.161 

4 

.143 

.083 

.023 

5 

.250 

.286 

.083 

.096 

.046 

6 

.286 

.332 

.288 

.023 

7 

.286 

.096 

.069 

8 

.429 

.249 

.048 

.046 

73 


APPENDIX  F:   NONPARAMETRIC  ANALYSIS 


EASE  OF  USE 


2 

3 

4 

Sub  j  . 

S 

R 

S 

R 

S 

R 

S 

R 

1 

3 

1 

6 

4 

4 

2 

5 

3 

2 

10 

2 

14 

4 

10 

2 

10 

2 

3 

3 

3 

15 

4 

0 

1 

2 

2 

4 

10 

2 

13 

4 

10 

2 

10 

2 

5 

10 

3 

15 

4 

4 

1 

7 

2 

6 

5 

3 

15 

4 

1 

1 

4 

2 

7 

10 

3.5 

10 

3.5 

0 

1.5 

0 

1.5 

8 

10 

2.5 

15 

4 

5 

1 

10 

2.5 

8 

E  R 
i=l 


20 


31.5 


11.5 


ID 


17 


2 


12 


x-  :  hk(k+l) 


k  /  n 

E     E   R.  . 

Lj=i\i=i  1D 


-  3n(k+l) 


12 


8(4) (4+1) 


[400  +  992.25  +  132.25  +  289]  -  3(8) (4+1) 


=  15.96* 


X0,05,3  "  7'82 
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SUITABILITY 


Subj. 

1 

2 

3 

4 

S 

R 

S 

R 

S 

R 

S 

R 

1 

4 

1 

6 

4 

5 

2.5 

5 

2.5 

2 

8 

1 

14 

4 

9 

2 

11 

3 

3 

15 

3.5 

15 

3.5 

0 

1 

5 

2 

4 

13 

3 

14 

4 

9 

1 

11 

2 

5 

9 

3 

14 

4 

3 

1 

4 

2 

6 

10 

3.5 

10 

3.5 

1 

1 

5 

2 

7 

5 

3 

5 

3 

5 

3 

0 

1 

8 

5 

1.5 

15 

4 

5 

1.5 

10 

3 

8      19.5 
Z  R.  . 
i=l  ^ 


30 


13 


17.5 


X2  =  8(4H5)  t380'25  +  900  +  169  +  306.25]  -  3(8)  (5) 


=  11.64* 


X0,05,3 


=  7.82 
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